Value Functions Factorization With Latent State Information Sharing in Decentralized Multi-Agent Policy Gradients

Value function factorization via centralized training and decentralized execution is promising for solving cooperative multi-agent reinforcement tasks. One of the approaches in this area, QMIX, has become state-of-the-art achieved best performance on StarCraft II micromanagement benchmark. However, monotonic-mixing per agent estimates QMIX known to restrict joint action Q-values it can represent, as well insufficient global state information single value estimation, often resulting suboptimality. To end, we present LSF-SAC, a novel framework that features variational inference-based information-sharing mechanism extra assist individual agents factorization. We demonstrate such latent sharing significantly expand power factorization, while fully still be maintained LSF-SAC through soft-actor-critic design. evaluate challenge outperforms several methods challenging collaborative further set extensive ablation studies locating key factors accounting its improvements. believe new insight lead local estimation deep learning algorithms. A demo video code implementation found at https://sites.google.com/view/sacmm.